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Summary 


A  group  of  20  experienced  sonar  operators  (SO)  was  compared  with  a  group  of  20 
non-operators  (NO)  on  a  computerized  word  decision  task.  The  subjects  had  to 
respond  quickly  and  accurately  in  determining  whether  a  word/words  presented 
visually  and/or  aurally  (at  the  same  time)  was/were  related  to  a  given 
category  (i.e.,  animals).  Response  speed,  decision  accuracy,  eund  recall  of 
words  related  and  unrelated  to  the  categories  were  recorded.  Results 
indicated  no  group  differences  in  any  of  the  dependent  measures,  thus 
experience  in  a  bimodal  input  situation  did  not  provide  sonar  operators  with  a 
performance  advantage  on  the  experimental  task.  All  40  subjects  performed 
faster  and  more  accurately  when  the  information  was  presented  visually,  and 
the  words  were  related  to  a  given  category.  Subjects  performed  about  as 
quickly  and  accurately  vdien  the  same  word  (related  to  the  category)  was 
presented  concurrently  in  both  modalities,  however,  recall  of  these  words  was 
significantly  higher  than  all  other  conditions.  Overall,  the  data  are 
consistent  with  other  studies  which  demonstrate  no  loss  and  some  gain  in  the 
bimodal  redundant  (same  word)  condition.  This  study  is  one  of  the  first  to 
demonstrate  an  enhamcement  in  higher  level  cognitive  performance  (such  as 
memory  recall)  in  a  bimodal  task. 
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introductiOT 


A  growing  body  of  literature  suggests  that  in  various  detection  and 
decision  tasks  that  "redundant"  relevant  information  in  two  modalities  is  as 
*  useful  or  more  beneficial  than  either  single  input  alone  (Leweindowski  &  Kobus, 

1969).  Such  results  have  been  demonstrated  using  siiqple  stimuli  as  lights  and 
*'  tones  (Nickerson,  1973)  as  well  as  more  complex  stimuli  as  sonar  signals 

(Kobus  et  al.,  1986),  letters  (Miller,  1982),  and  words  (Leweindowski,  Hursh,  & 
Kobus,  1985).  A  long-standing  interest  in  the  processing  of  dual  signals  has 
been  evidenced  by  sonar  operators  and  research  scientists  working  with  sonar 
systems.  While  sonar  systems  such  as  those  used  by  the  U.S.  Navy  have  changed 
considerably  over  the  years,  modern  systems  continue  to  provide  operators  with 
concurrent  visual  and  auditory  information.  At  times  that  information  may  be 
simple  or  complex,  contain  alphanumeric  symbols  or  not,  and  be  redundant  or 
conflictual.  The  ability  to  make  cognitive  decisions  on  the  basis  of  incoming 
bimodal  information  under  these  various  conditions  warrants  ongoing 

investigation.  A  sinple  question  remains:  "What  is  the  best  information 
delivery  condition  for  the  sonar  operator?" 

Research  on  bimodal  processing  on  Navy  sonar  systems  has  consistently 
indicated  a  value  in  providing  two  inputs  versus  one  to  an  operator  (Kobus  & 
Lewandowski,  1986;  Lewandowski  &  Kobus,  1985).  Yet,  with  ever-improving 
computer  graphics,  the  visual  component  of  the  information  delivered  has  been 
enhanced  and  complicated.  It  has  been  suggested  that  the  enhanced  visual 
displays  may  be  best  if  operated  in  isolation,  and  that  the  auditory 
information  may  actually  be  deleterious  to  overall  operator  performance. 

The  question  then  becomes:  "Does  the  auditory  channel  provide  any  useful 
information  above  and  beyond  the  visual  channel  alone?"  Thus  far,  research 
has  suggested  that  redundant  information,  delivered  concurrently  in  two 
V  channels,  tends  to  have  a  positive  effect  on  performance,  a  finding  which  has 

held  across  a  range  of  context  and  stimulus  types.  For  example,  Lewandowski 
and  KoIdus  (1985)  found  that  target  detection  thresholds  were  reduced  when  an 
equivalent  sonar  target  was  concurrently  presented  in  both  the  visual  and 
auditory  channels.  Lewandowski,  Hursh,  and  Kobus  (1985)  extended  this  finding 
in  an  analogous  task  by  presenting  words  to  subjects  in  either  visual  or 
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auditory  channels,  or  in  both  cheinnels.  Subjects  were  asked  to  decide  if  a 
word  or  words  belonged  to  a  given  category.  When  the  seime  word  was  presented 
in  each  channel,  choice  response  time  (RT)  was  as  fast  or  faster  than  either 
single  modality.  This  study  demonstrated  that  the  bimodal  facilitation  effect 
can  apply  to  more  complex  and  meaningful  stimuli  than  previously  shown,  in  a 
third  study,  Kobus  and  Lewandowski  (1986)  had  shown  that  on  a  simulated  sonar 
task,  bimodal  facilitation  for  redundant  stimuli  occurred  across  dependent 
measures.  They  found  that  the  bimodal  redxmdant  target  condition  produced 
faster  and  more  accurate  responses  than  any  other  condition.  Additionally, 
the  bimodal-different  condition  (conflicting  information  in  the  two 
modalities)  produced  the  slowest  and  least  accurate  results. 

The  current  study  was  a  combination  of  the  above  paradigms.  This  study 
involved  a  semantic  choice  RT  task  in  which  category  words  were  presented  in 
various  unimodal  and  bimodal  conditions.  Since  displays  now  require 
nigher-level  processing  of  complex  information,  and  sonar  operators 
concurrently  engage  in  verbal  communication  with  the  Conmand  Information 
Center  (CIC),  this  study  examined  certain  cognitive  performance  of  subjects 
while  receiving  verbal  stimuli  in  one  or  two  modalities.  Therefore,  in 
addition  to  RT,  we  investigated  semantic  decision  accuracy  and  memory  recall. 
Sonar  operators  were  compared  with  non-operators,  and  presentation  conditions 
were  contrasted  to  determine  the  best  overall  condition  for  presenting 
information.  The  bimodal-redundant  condition  was  expected  to  be  the  best 
performance  condition,  and  sonar  operators  were  expected  to  out  perform 
non-operators,  at  least  in  certain  conditions. 

Method 

Subjects 

Subjects  were  20  active-duty  sonar  operators  (SO  group)  from  the 
Anti-Submarine  Warfare  (ASW)  Base  in  San  Diego,  California,  and  20 
non-operators  (NO  group),  solicited  from  an  urban  population  such  that  the 
groups  were  comparable  in  age,  education,  mental  ability,  and  socioeconomic 
status.  The  sonarmen  each  had  at  least  four  years  of  sonar  experience,  and 
most  were  sonar  instructors  who  were  considered  to  be  high  level  performers. 
The  average  age  of  these  subjects  was  29.2  years  (ranging  from  23  to  40 
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years).  All  subjects  would  be  considered  to  be  in  a  middle  class 
socioeconomic  category  based  on  occupational  classification.  IQ  tests  were 
not  administered,  but  previous  research  with  this  population  revealed  average 
to  above  average  mental  abilities  (Lewandowslti ,  Kobus,  &  Flood,  1987).  The 
*  range  of  this  group's  educational  level  was  from  14  to  20  years  (M  =  14.2 

yrs.).  Visual  and  auditory  acuity  of  these  subjects  was  routinely  chec)<ed, 

^  and  were  within  normal  (or  corrected  to  normal)  limits.  The  NO  group 

consisted  of  20  men  ranging  in  age  from  21  to  43  years  (M  -  29.5  yrs.)  These 
men  had  no  sonar  experience  and  no  specialized  training  in  visual  or  auditory 
detection.  On  the  basis  of  auditory  and  visual  pre-testing,  vision  and 
hearing  were  determined  to  be  adequate  for  the  experiment.  These  subjects 
were  all  considered  to  be  in  a  middle  class  category,  based  on  occupational 
classification,  having  a  slightly  higher  mean  level  of  education  (M  -  16.7 
yrs.).  Mental  ability  of  this  group  was  expected  to  be  above  average. 

Apparatus 

The  word  decision  task  was  presented  via  an  APPLE  II'*’  computer.  Visual 
information  was  displayed  on  a  Zenith  1200  monochrome  CT,  while  auditory  words 
were  presented  by  a  "supertalker"  speech  processor  (Mountain,  Inc.)  through 
monaural  headphones  (Maico).  RT  responses  were  recorded  by  depressing 
specified  keys  on  the  computer  keyboard.  The  computer  task  consisted  of  18 
blocks  (categories)  of  words  (three  practices  followed  by  15  test  blocks). 
Each  block  contained  nine  different  presentations  (trials)  of  words  that  were 
either  related  or  unrelated  to  a  particular  category.  An  example  of  a  block 
of  trials  is  presented  in  Table  1.  Note  that  each  trial  represents  a 
different  presentation  condition,  thus  each  block  has  one  trial  for  each  of 
the  nine  conditions.  Within  a  given  block,  the  trials/conditions  were 
randomly  ordered.  All  subjects  received  the  same  order  of  blocks  and  trials. 

f  The  words  used  in  the  task  were  selected  from  the  Battig  and  Montague 

(1969)  category  norms.  They  are  all  one-syllable  words  of  relatively  high 
frequency  (A  to  AA)  based  on  the  Paivio,  Yuille,  and  Madigan  (1968)  frequency 
tables.  Words  across  conditions  were  equated  to  words  appearing  in  the 
auditory  conditions  according  to  frequency,  imageability,  and  length. 
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Table  1. 

A  Sample  Block  of  Words 

Category 

Condition 

Visual  word 

Auditory  word 

( 

1 

Parts  of  a  | 

Auditory-U 

1 

1 

1 

1 

1 

1 

SMILE 

Room  1 

Bimodal-di f  f e  rent-R 

1 

WALL  1 

FLOOR 

1 

Visual-R 

1 

RUG  1 

1 

Bimodal-di fferent-U 

1 

LAMP  ) 

CHEESE 

1 

Auditory-R 

1 

1 

DESK 

1 

Bimodal-same-R 

1 

CHAIR  1 

CHAIR 

1 

Bimodal-di fferent-U 

1 

ZOO  1 

SLEIGH 

1 

Bimodal-same-U 

1 

STREET  1 

STREET 

1 

1 

_ L 

Visual-U 

1 

1 

WING  1 

1 

_ L 

R  =  Related  to  category 
U  =»  Unrelated 


Procedure 


All  subjects  participated  voluntarily,  and  gave  written  consent. 
Piloting  ensured  the  task  ran  smoothly,  and  that  words  presented  aurally  could 
be  identified  accurately  90%  of  the  time.  Following  an  explanation  of  the 
study,  subjects  completed  a  short  backgroup  questionnaire.  Next,  subjects 
were  screened  for  visual  or  auditory  deficits.  Visual  memory  and  auditory 
memory  pretests  were  then  administered  in  counterbalanced  order.  The  visual 
memory  pretest  consisted  of  a  printed  list  of  14  unrelated  words.  Subjects 
had  15  seconds  to  study  the  list,  and  then  immediately  recall,  in  writing,  as 
many  words  as  possible.  The  auditory  pretest  consisted  of  14  unrelated  words 
presented  in  15  seconds  on  an  audio  tape  (pre-recorded).  Again,  the  subject 
had  to  immediately  recall  in  writing  as  many  words  as  possible.  The  pretest 
prepared  subjects  for  the  experimental  task,  and  screened  for  possible  short¬ 
term  memory  deficits. 
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Subjects  were  then  seated  in  front  of  the  computer  where  they  read 
directions  from  the  CRT.  The  task  was  also  explained  by  the  experimenter. 
Subjects  were  instructed  that  for  each  block  of  trials,  a  category  name  would 
appear  on  the  screen.  The  words  to  be  presented  would  either  belong  to  that 
category  or  not.  If  any  word  in  a  trial  was  from  that  category,  they  should 
immediately  press  the  "YES"  key;  if  the  words  presented  did  not  fit  the 
category,  they  should  press  the  key  designated  "NO."  All  subjects  used  their 
right  hand  fingers  to  respond.  The  index,  middle,  eu\d  ring  finger  were 
lightly  rested  on  the  keys  to  minimize  movement  time.  A  third  key  between  the 
two  response  keys  was  depressed  whenever  the  subject  was  ready  to  initiate  a 
new  trial. 

Once  the  subject  made  RT  responses  on  all  nine  trials  of  a  block,  the 
computer  informed  them  that  the  block  was  complete.  They  then  had  to  write 
the  words  they  remembered  from  that  block  on  a  response  sheet  in  front  of 
them.  The  instructions  emphasized  making  the  fastest  decision  possible,  and 
then  moving  on  to  the  next  trial,  thus  not  allowing  time  to  rehearse  words 
already  presented.  In  this  sense,  the  word-recall  component  was  much  like  an 
incidental  memory  test.  Once  subjects  completed  the  three  practice  and  15 
test  blocks  (about  40  minutes  to  complete  all  162  trials),  they  were  debriefed 
by  the  experimenter. 


Results 

The  dependent  measures  for  this  task  were;  1)  response  time  (RT), 
recorded  in  milliseconds,  with  a  median  RT  determined  for  each  condition; 
2)  accuracy,  as  measured  tiy  percentage  of  correct  choices;  and,  3)  recall,  as 
measured  by  percentage  of  words  recalled  per  condition.  Only  eight  of  the 
nine  trials  (each  block  had  one  practice  trial)  were  entered  into  the 
analyses.  The  design  thus  became  a  2.42  split  plot  analysis  (see  Kirk,  1968) 
with  the  between  subject  variable  Group  (SO  and  NO)  and  the  within  variables 
Modality  (visual,  auditory,  bimodal-same  word,  and  bimodal-dif ferent  words) 
and  Relatedness  (related  or  unrelated  to  the  category). 

Initial  analyses  of  the  data  included  three  analyses  of  variance  (ANOVAs) 
with  repeated  measures.  Each  analysis  was  performed  on  one  of  the  dependent 
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measures,  and  encompassed  the  two  groups  and  four  presentation  conditions  with 
word  Relatedness  nested  within.  These  analyses  yielded  no  group  differences 
on  any  of  the  dependent  measures.  Hierefore,  the  group  data  were  collapsed 
for  all  further  euialyses.  The  mean  data  (collapsed  across  groups)  are 
presented  in  Table  2.  The  ANOVA  of  RT  data  did  show  significant  effects  for 


Table  2.  Mean  Performances  of  All  Subjects  Across  Conditions 


Measure  Conditions 


Vi  sual 

1 

Auditory 

Bimodal-same 

1 Bimodal-different 

1 

R 

U 

1 

R 

U 

R  U 

1 

U 

RT 

804 

954 

I 

1 

1195 

1270 

850  1063 

1 

1  970 

1496 

%  Correct 

96 

93.2 

1 

77.7 

91.7 

94  93.3 

1  91.6 

92 

%  Recall 

66.7 

36.5 

1 

_L 

64 

29 

81.3  37.8 

1  67.2 

25.5 

RT  =  Reaction  Time 
R  >•  Related  to  category 
U  =  Unrelated  to  category 


both  Modality  and  Relatedness  plus  an  interaction  between  the  two 
(all  p  <  .001).  Post  hoc  comparisons  indicated  that  RT  was  faster  \dien  words 
were  related  to  a  category,  and  that  performance  was  faster  in  the  visual 
mode,  followed  by  the  bimodal-same  condition.  The  interaction  seemed  to  be 
due  to  an  unusually  slow  RT  in  the  auditory  mode  when  the  word  was  related  to 
the  category,  and  an  extremely  slow  RT  in  the  bimodal-different  condition  when 
the  words  were  unrelated.  The  analysis  of  accuracy  data  resulted  in 
significant  main  effects  for  Modality  and  Relatedness  as  well  as  their 
interaction  (p  <  .001).  Subjects  were  generally  more  accurate  in  the  related 
conditions  except  when  the  word  was  presented  aurally.  Once  again, 
performance  in  the  visual  mode  exceeded  that  of  the  other  conditions,  followed 
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closely  by  the  bimodal-same  condition.  This  time,  however,  accuracy  was 
poorest  in  the  auditory  mode,  particularly  when  the  word  was  related  to  the 
category. 

•  The  analyses  of  recall  data  yielded  significant  effects  for  Modality  and 

Relatedness  (p  <  .001).  Word  recall  was  generally  higher  for  related  words, 
“  and  was  significantly  superior  in  the  bimodal-same  condition  for  related 

words.  Recall  for  related  words  in  the  other  conditions  did  not  differ  from 
one  another. 

Correlational  analyses  relating  the  dependent  measures  across  conditions 
yielded  few  interesting  results.  Within  a  measure  (i.e.,  RT),  correlations 
tended  to  be  high,  whereas,  between  measures  correlations  tended  to  be  low  to 
moderate.  Of  most  interest,  was  a  chec)c  for  speed-accuracy  trade-offs  between 
RT  and  accuracy.  The  correlational  data  had  shown  little  relationship  between 
these  two  performances,  perhaps,  because  accuracy  was  consistently  high. 

Discussion 

The  results  of  this  study  are  generally  consistent  with  similar  prior 
investigations  ( Lewandowski ,  Hursh,  &  Kobus,  1985).  Overall,  the  data 
indicated  that  subjects  respond  faster  and  more  accurately  to  words  presented 
in  the  visual  modality.  They  are  nearly  as  efficient  when  information  is 
redundant  and  related  to  a  category  in  the  bimodal-same  condition.  The 
slightly  longer  response  time  (45  ms)  probably  reflects  the  longer 
presentation  time  of  an  auditory  word  (vs.  visual  presentation  time).  Despite 
this  slightly  longer  RT,  subjects  are  able  to  perform  at  a  highly  accurate 
level,  and  more  importantly,  are  able  to  recall  the  greatest  percentage  of 
,  words  in  the  bimodal-same  condition.  If  one  assumes  that  the  recall  conponent 

reflects  the  highest  level  of  cognitive  processing  measured  by  the  task,  then 
<  one  would  have  to  say  that  the  bimodal-same  condition  is  of  some  benefit. 

Sonar  operators  are  required  to  make  a  variety  of  complex  decisions  regarding 
incoming  signals,  yet  most  laboratory  studies  have  investigated  only  simple 
lights  and  tones.  The  present  study  has  examined  bimodal  processing  cf 
meaningful  (complex)  information  requiring  higher  lever  cognitive  activities 
such  as  decision  making  and  memory. 
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The  results  support  the  previous  work  of  Lewandowski,  Hursh,  and  Kobus 
(1985)  by  demonstrating  that  the  bimodal-same  condition  is  comparable  to  the 
better  single  modality  (in  this  case  visual)  in  RT  for  a  group  of  40  subjects. 
Similarly,  the  present  study  is  consistent  with  results  of  Kobus  euid 
Lewandowski  (1986)  who  found  that  sonar  target  detection  was  performed  faster 
and  more  accurately  when  the  same  information  was  delivered  to  both  modalities 
rather  than  one  modality  or  conflicting  information  in  both.  This  is 
essentially  the  pattern  of  results  found  in  the  present  bimodal  word  study. 
Besides  replicating  previous  research,  this  study  extends  this  line  of 
investigation  by  demonstrating  the  facilitation  of  memory  in  the  bimodal-same 
condition.  This  enhancement  of  a  cognitive  performance  is  promising,  and 
suggests  the  need  for  further  research  on  the  influence  of  bimodal  information 
presentation  on  other  cognitive  performances.  It  still  appears  that  in 
bimodal  tasks,  regardless  of  the  stimuli  and  paradigms  employed,  the 
bimodal- redundant  condition  is  a  desirable  one  for  overall  performance,  this 
would  also  appear  to  be  the  case  for  sonar  operation.  Designers  of  sonar 
systems  may  want  to  consider  ways  of  providing  dual  information  (using  words 
or  symbols  to  indicate  important  data). 

The  fact  that  no  group  differences  were  found  on  any  of  the  dependent 
measures  is  a  little  surprising.  One  might  expect  that  the  bimodal  task 
experience  of  sonar  operators  would  be  reflected  in  better  performance  than 
that  of  non-operators.  Perhaps,  the  word  task  was  different  enough  from  sonar 
and  equally  novel  for  both  groups  of  subjects,  such  that  no  differences  were 
found.  Similar  results  (no  group  differences)  have  been  found  in  other  sonar 
simulation  studies  using  comparison  groups  (Kobus,  Lewandowski,  &  Flood,  1987; 
Lewandowski,  Kobus,  Flood,  &  Hoyer,  1988).  A  next  step  would  be  to  develop  a 
research  paradigm  that  more  closely  simulates  present  sonar  operation,  yet 
still  allows  for  experimental  manipulation  of  bimodal  processing  at  higher 
cognitive  levels. 
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operators  (NO)  on  a  computerized  word  decision  tasls.  The  subjects  had  to  respond  quiclsly 
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recorded.  Results  indicated  no  group  differences  in  any  of  the  dependent  measures,  thus 
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to  a  given  category.  Subjects  performed  about  as  quickly  and  accurately  when  the  same 
word  (related  to  the  category)  was  presented  concurrently  in  both  modalities,  however, 
recall  of  these  words  was  significantly  higher  than  all  other  conditions.  Overall,  the 
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and  some  gain  in  the  biroodal  redundant  (same  word)  condition.  This  study  is  one  of  the 
first  to  demonstrate  eui  enhancement  in  higher  level  cognitive  performance  (such  as  memory 
recall)  in  a  bimodal  tas)c. 


